Practical Case Of Multi-platform Synchronization And Cross-store Data Analysis Of Shopee Taiwan Store Group

1. project background and goals

- project target: shopee taiwan station store group (pseudonym a store group), with the goal of simultaneous launch on multiple platforms (shopee, pchome, open-air) and cross-store data analysis.
- target indicators: peak daily orders of 8,500 orders, peak api calls of 350 calls/min, average response time < 200ms.
-technical points: stable vps/host deployment, domain name and dns strategy, cdn accelerated static resources, ddos protection and log analysis.
- core requirements: ensure multi-store synchronization without conflicts, real-time consistency of cross-store inventory, and reduce order delays and api failure rates.
- risk points: high concurrent api calls cause the host cpu and bandwidth to soar, and unexpected traffic triggers ddos protection accidental killing.

2. infrastructure design (vps and host layout)

- the front-end cdn layer uses edge caching to reduce bandwidth pressure on the origin site, and static files and images go through cdn.
- the application layer uses 3 application vps: specification example 4 vcpu / 8gb ram / 80gb nvme / 1gbps.
- cache layer uses redis (example: 2 vcpu / 4gb ram), saving sessions and inventory locks.
- independent master-slave database: 8 vcpu/32gb ram/1tb nvme in the master database, and the slave database is used for reports and cross-store analysis and query.
- load balancing: use nginx reverse proxy + health check, or cloud lb for traffic distribution and failover.

3. domain name and dns strategy

- use wildcard for the main domain, such as *.shop.example.com, which facilitates multi-store subdomain management.
- dns ttl is set to 300 seconds to take effect quickly when switching occurs.
- dns provider supports geo dns and health detection to avoid single points of failure.
- use dns load balancing to distribute to vps in different availability zones, and combine it with cdn for secondary caching.
- record backup: configure mx/caa/txt and other records to improve domain name security and email delivery rate.

4. cdn and static resource acceleration strategy

- all static images and js/css are uploaded to cdn, reducing origin site io and network bandwidth requirements.
- cache invalidation strategy: long-term caching of product images, short caching or no caching of product prices/inventory (pulled through api).
- cdn configures https and enables http/2 to improve concurrency performance.
- upstream return-to-origin speed limit and concurrency control prevent instantaneous return-to-origin traffic from overwhelming the backend.
- cdn saves about 70% of origin site traffic on average every day, and the peak return-to-origin bandwidth is reduced from 400mbps to 120mbps (sample data).

5. ddos defense and security strategy

- use cloud ddos protection (such as cloudflare / aws shield) for site-wide traffic filtering.
- edge waf rules block common crawlers and abnormal api request patterns.
- current limiting strategy: token verification and qps limit for external platform api (for example, 5 qps per second per store).
- black and white lists are combined with automatic blocking of abnormal ips, and logs are stored for 30 days to facilitate traceability.
- drills and monitoring: simulate 10gbps attacks every quarter to verify automatic expansion and switching logic.

6. key points of multi-platform synchronization implementation

- synchronization method: use message queue (rabbitmq/kafka) decoupling to ensure reliable retry of multi-platform tasks.
- idempotent design: each synchronization operation has a unique serial number to avoid repeated inventory deductions.
- api current limiting: create dynamic entry queue for rate limiting of shopee and other platforms.
- time window: batch delayed listing and offloading are used during peak periods to reduce instantaneous writing pressure.
- log and compensation mechanism: failed tasks are written into the compensation table, and inventory differences are corrected by running batch replenishment at night.

7. cross-store data analysis and database configuration examples

- the analysis library uses a copy of the slave library to avoid affecting the primary write performance, and the backup cycle is one copy per day.
- use olap to extract to clickhouse/elasticsearch for real-time reporting and search.
- indicator examples: store conversion rate, inventory turnover, order delay on each platform (ms), etc.
- data retention: order details are retained for 2 years, and hot data is stored in redis for 90 days.
- monitoring: prometheus + grafana monitors cpu/ram/network/io, and automatically configures alarm thresholds.

8. real configuration and performance data table (example)

the following is an example of peak monitoring and server configuration of store group a on 2025-10-12:

components	specifications/description	peak indicator
application servers (3 units)	4 vcpu/8gb/80gb nvme/1gbps	average cpu 65% / single bandwidth 90mbps
main database	8 vcpu/32gb/1tb nvme/1gbps	primary write latency 12ms/qps 320
redis cache	2vcpu/4gb	hit rate 92% / latency 1.8ms
cdn	edge caching + https	origin site return bandwidth 120mbps